Papers with English language models
Investigating Aspect Features in Contextualized Embeddings with Semantic Scales and Distributional Similarity (2024.starsem-1)
Copied to clipboard
| Challenge: | Aspect is a linguistic category describing how actions and events unfold over time. |
| Approach: | They propose to use semantic projections to examine whether the vector dimensions of annotated verbs reflect human linguistic distinctions. |
| Outcome: | The proposed models encode the aspects of stativity, durativity and telicity in most of their layers, while durativité is the most challenging feature. |
MuLan: A Study of Fact Mutability in Language Models (2024.naacl-short)
Copied to clipboard
| Challenge: | Pretrained and large language models encode factual knowledge, but factual information changes over time and mutates with the passage of time. |
| Approach: | They propose to use a model to evaluate the ability of English language models to anticipate time-contingency by comparing their models to a benchmark model. |
| Outcome: | The proposed model can predict the president of a country or the winner of sa championship in time, but it is difficult to update them due to their mutability. |
Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance (2024.findings-acl)
Copied to clipboard
| Challenge: | Despite being the cornerstone of BPE, the importance of compression in the tokenization process is still unclear. |
| Approach: | They argue for the theoretical importance of compression in the tokenization process . they also demonstrate the empirical importance of compressing tokenizers for downstream success of pre-trained language models. |
| Outcome: | The proposed method can be viewed as 0-gram language modeling where equal probability is assigned to all tokens. |
Evaluating Moral Beliefs across LLMs through a Pluralistic Framework (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Proper moral beliefs are fundamental for language models, yet assessing these beliefs poses a significant challenge. |
| Approach: | They propose a framework to evaluate the moral beliefs of four large language models . they use a dataset containing 472 moral choice scenarios in Chinese . |
| Outcome: | The proposed framework evaluates the moral beliefs of four large language models. |
Controlled Evaluation of Grammatical Knowledge in Mandarin Chinese Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has shown that structural supervision helps English language models learn generalizations about syntactic phenomena such as subject-verb agreement. |
| Approach: | They train LSTMs, Recurrent Neural Network Grammars, Transformer language models, and Transformer-parameterized generative parsing models on Mandarin Chinese datasets. |
| Outcome: | The proposed models learn aspects of Mandarin Chinese grammar that assess syntactic and semantic relationships. |
Social Biases in NLP Models as Barriers for Persons with Disabilities (2020.acl-main)
Copied to clipboard
| Challenge: | toxicity prediction and sentiment analysis models perpetuate undesirable social biases from the data on which they are trained. |
| Approach: | They propose to use toxicity prediction and sentiment analysis to examine whether NLP models perpetuate undesirable biases towards mentions of disability. |
| Outcome: | The proposed models contain undesirable biases towards mentions of disability in two English language models. |